The current methods of distributing multimedia data over the wireless Internet to mobile clients are constrained by existing wireless bandwidth, and the real-time decoding, processing and displaying of multimedia content with limited hardware capabilities. These hardware limitations include slow CPUs, high memory latencies, slow drawing capabilities and the absence of YUV to RGB conversion in the hardware.
Video and audio playback exist on certain cell phone handsets, but this technology is embedded and takes advantage of low-level hardware processing to enable the performance required for media playback. Through embedded media players, several cell phone handsets and handheld computers can play either streamed video or audio. Popular digital video encoding standards for some handsets are H263 and MPEG4. The audio codecs, MP3, AMR and AAC, are also typically supported on some mobile handhelds. Newer video codecs, like H264 could be used for video transmission to cell phones, but would require clients systems with fast memory access for their motion compensation methods.
Embedded streaming media players rely on firmware integration to take advantage of the multitasking capabilities of cell phone handsets. At the time of this writing, most cell phones cannot support multimedia playback because they are only capable of supporting one or a few concurrent processing threads. On handsets that have embedded media players, video is also limited to very low frame rates and the bandwidth available for streaming in North America is low, varying from 2-3 kybtes/second to ISDN speeds of 64 Kbits. European countries and Japan currently offer 3G network connection speeds, varying from 64 kbits-300 Kbits, and offer more technologically advanced cell phones with embedded media players that can achieve higher video frame rates. For limited per usage periods and some, EV-DO (Evolution Data Optimized) networks can also provide these higher speeds over local CDMA networks.
Decoders for complex video codecs which support highly scalable MPEG4 video, and more complex, CD quality music, audio codecs like AAC, require multiple parallel processes and fast processing. Mathematical algorithms, as in Fan and Madisetti, designed for the high number of floating point samples for higher end MPEG4 and AAC, which requires a sample rate approximately 36,000 floating point calculations/second, are intended to run on specialized chips. Even at lower and very low bitrates, where MPEG4 is more efficient than its predecessors, MPEG4 software players depend on PC multitasking or hardware APIs for efficient processing to draw video frames.
Currently, device independent Java (J2ME) offers two standard configurations on mobile clients. The Connected Limited Device Configuration (CLDC) is prevalent in the J2ME world, and powers cellular phones, pagers, PDAs, and other handheld devices. A variant of CLDC, Connected Device Configuration (CDC) targets more powerful devices, such as home appliances, set-top boxes, and Internet TVs. The second configuration, Mobile Information Device Profile (MIDP), runs on top of the CLDC, and several profiles run on top of CDC.
Java players for cell phones like the Oplayo MVQ player exist, but implementations of true, platform independent, MPEG4 Java decoders that will play video on cell phones are not known. More efficient methods, such as U.S. Pat. No. 5,699,121, do not rely on DCT motion compensation, and propose pattern marching to identify regions in the motion residual signal that have not been accurately reproduced and to correct them using a pattern library. But again, this approach does not use MPEG4 video encoding.
Similarly, although Java decoders exist that play MP3 ringtones on cell phones, no Java players are known that will play AAC. In fact, many of the newer IDCT algorithms are targeted more towards customized logic chips that only do IDCT (composed of many simple pipelined instructions as opposed to a few more complex ones).
Bit streaming is the standard method of transmitting audio or video to cell phones over wireless networks. Streamed bits are buffered and then decoded or entire video files are downloaded or proportionately cached, or as in progressive, http streaming downloading to a point where complete, continuous playback is deemed possible. In the case of wireless networks, bit streaming in terms of audio/video content is usually is done over a non-reliable transport like UDP and requires a lot of error correction and duplication of content (extra stream correction data). Advances in these transmission methods do propose more sophisticated means of reserved bandwidth, such as AROMA. Alternatives to streaming methods have been proposed for the transmission of video as objects through pre-fetched lists Waese et al., U.S. Pat. No. 6,286,031, which are similar to downloading pre-fetched lists of SMIL objects, and instant or scheduled notification file downloading Stumm, U.S. Pat. No. 5,768,528. However, these do not address specific continuity and deployment issues for wireless multimedia transmission and concurrent playback on a limited-tasking cell phone handsets.
Streaming also requires client processing for significant error correction in video decoding, adaptive encoding for varying channel bitrates and in cellular networks. It also requires the use of cellular MMS multimedia protocol.
The object of the current invention is to solve one or more of the drawbacks in existing methods discussed above, or to provide other improvements to the art.